Skip to content

feat(coverage-floor): add test_command input for layered Python repos#63

Open
topcoder1 wants to merge 1 commit into
mainfrom
feat/coverage-floor-test-command
Open

feat(coverage-floor): add test_command input for layered Python repos#63
topcoder1 wants to merge 1 commit into
mainfrom
feat/coverage-floor-test-command

Conversation

@topcoder1
Copy link
Copy Markdown
Owner

Summary

Adds optional test_command: input to the coverage-floor.yml reusable workflow. When set, it replaces the default language-specific install + pytest/go-test/npm-test invocation; the caller's command is expected to produce the same coverage artifact the workflow already parses (coverage.json / cov.out / coverage/coverage-summary.json).

Mirrors the same input on tests-runner.yml.

Concrete trigger

whois-api-llc/wxa_marketing (today). Layered Python project:

  • Root pyproject.toml defines the marketing-engine package with httpx, pytest-asyncio, aiosqlite in [project.optional-dependencies].test
  • Tests live under backend/ and import marketing_engine.* (and httpx in conftest)
  • backend/requirements.txt only has the test runner deps

The default Python path falls into [[ -f requirements.txt ]] → pip install -r requirements.txt && pytest from backend/, which doesn't install the engine package itself, so from httpx import AsyncClient in conftest.py fails with ModuleNotFoundError. Same failure mode predicted in CLAUDE.md lessons ("Coverage-floor reusable's naïve install ... misses repo-specific test-environment quirks").

After this lands, the caller can write:

with:
  language: python
  working_directory: backend
  test_command: |
    python -m venv .venv
    .venv/bin/pip install -q --upgrade pip
    .venv/bin/pip install -q -e "..[test]" pytest-cov
    .venv/bin/pytest --cov --cov-report=json --quiet || true

Same pattern already de-risked on tests-runner.yml — in flight today as whois-api-llc/wxa_marketing#25.

Design

Per-branch short-circuit inside the existing case "$LANG_TYPE" block rather than wrapping the whole case. Reasons:

  1. Preserves the per-language coverage-file existence check — the workflow still errors clearly if the caller's command produces no coverage file ("pytest-cov did not produce coverage.json"). The check is the contract.
  2. Preserves the per-language MEASURED jq expression — different runners emit different JSON shapes (.totals.percent_covered vs .total.lines.pct vs Go's text format).
  3. Minimal diff — 3 short-circuit blocks (4 lines each), no restructuring.

Test plan

  • python -m yaml.safe_load clean
  • bash -n clean on the measure step's embedded run script
  • All three case branches short-circuit when INPUT_TEST_COMMAND is non-empty
  • Coverage-file existence checks still enforce the right artifact gets produced
  • Defaults preserved: empty test_command (the default) keeps existing auto-detection behavior unchanged on every repo that doesn't pass it
  • Self-test on this repo (.coverage-floor at root, selftest/sample.py at 99% target) stays green

Auto-merge rationale: in-tree change to .github/workflows/coverage-floor.yml — high-risk surface per fleet policy (.github/workflows/**). Diff is purely additive (1 new input declaration, 3 short-circuit blocks); no permission/secret changes; no behavior change for callers that don't set the input. Per the cheat-sheet self-test: worst case is "callers measure coverage differently for one cycle, git revert and we're back" — auto-merge territory. Bot's regex decides.

Codex pre-review: 44-line additive diff in a single existing step. Skipping per the <50-line carve-out.

🤖 Generated with Claude Code

Adds optional `test_command:` input to the coverage-floor.yml reusable
that lets callers replace the default language-specific install + test
invocation with a custom shell command. The override MUST produce the
coverage file the workflow expects (`coverage.json` for python,
`cov.out` for go, `coverage/coverage-summary.json` for js); the existing
file-existence checks + MEASURED extraction at the end of each case
branch still run.

Mirrors the equivalent `test_command:` input already supported by
tests-runner.yml. Same use case both there and here: layered repos
where the default `uv sync` / `pip install -r requirements.txt` path
can't pick up the right package + extras.

Concrete trigger: whois-api-llc/wxa_marketing (today). Project layout
is root `pyproject.toml` for the `marketing-engine` package (with httpx
+ pytest-asyncio + aiosqlite in `[project.optional-dependencies].test`)
+ tests under `backend/` that import `marketing_engine.*`. The default
Python path falls into `[[ -f requirements.txt ]] → pip install -r
requirements.txt` from `backend/`, which doesn't install the engine
package, so `from httpx import AsyncClient` in conftest.py fails.

After this lands, the caller can override with e.g.:

  with:
    language: python
    working_directory: backend
    test_command: |
      python -m venv .venv
      .venv/bin/pip install -q --upgrade pip
      .venv/bin/pip install -q -e "..[test]" pytest-cov
      .venv/bin/pytest --cov --cov-report=json --quiet || true

Same pattern already de-risked on tests-runner.yml (in wxa_marketing#25,
in flight today).

## Test plan

- [x] `python -m yaml.safe_load` clean
- [x] `bash -n` clean on the measure step's embedded run script
- [x] All three case branches (python/go/js) short-circuit on
      `INPUT_TEST_COMMAND` set; coverage-file existence checks still
      enforce the right artifact gets produced
- [x] Defaults preserved: empty `test_command` keeps the existing
      auto-detection behavior on every repo that doesn't pass it
- [ ] Self-test on this repo (selftest/sample.py) stays green
      (`.coverage-floor` at root still seeds 99% for the self-test)

**Auto-merge rationale:** in-tree change to a reusable workflow under
`.github/workflows/**` — high-risk surface per fleet policy. Diff is
additive (1 new input declaration, 3 short-circuit blocks inside an
existing measure step); no permission/secret changes, no behavior
change for callers that don't set the input. Per the auto-merge
cheat-sheet self-test: worst case is "callers measure coverage
differently for one cycle, git revert and we're back" — auto-merge
territory. Bot's regex decides.

**Codex pre-review:** trivial — 44-line additive diff in a single
existing step. Skipping per <50-line carve-out.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

Coverage Floor — mode: enforce

metric value
measured 100.0%
floor (current) 99.0%
target 100.0%
last bumped 2026-05-12

@claude
Copy link
Copy Markdown

claude Bot commented May 13, 2026

No issues found. Additive diff is clean: eval "$INPUT_TEST_COMMAND" uses correct env-var indirection (not direct YAML interpolation), coverage-file existence checks still enforce the caller contract, and default behavior is unchanged for callers that don't set the input.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant